Portable holographic user interface for an interactive 3D environment

ABSTRACT

Disclosed are a method and corresponding apparatus to enable a user of a display system to manipulate holographic objects. Multiple holographic user interface objects capable of being independently manipulated by a user are displayed to the user, overlaid on a real-world view of a 3D physical space in which the user is located. In response to a first user action, the holographic user interface objects are made to appear to be combined into a holographic container object that appears at a first location in the 3D physical space. In response to the first user action or a second user action, the holographic container object is made to appear to relocate to a second location in the 3D physical space. The holographic user interface objects are then made to appear to deploy from the holographic container object when the holographic container object appears to be located at the second location.

BACKGROUND

Virtual reality (VR) and augmented reality (AR) visualization systemsare starting to enter the mainstream consumer electronics marketplace.AR Head-Mounted Display (HMD) devices (“AR-HMD devices”) are onepromising use of such technology. These devices may include transparentdisplay elements that enable a user to see virtual content transposedover the user's view of the real world. Virtual content that appears tobe superimposed over the user's real-world view is commonly referred toas AR content. Displayed AR objects are often referred to as“holographic” objects. VR and AR visualization systems can provide userswith entertaining, immersive three-dimensional (3D) virtual environmentsin which they can visually (and sometimes audibly) experience thingsthey might not normally experience in real life.

SUMMARY

The techniques introduced here enable a user of a display system tomanipulate holographic objects. In some embodiments, the technique isimplemented in an HMD device. In some embodiments, multiple holographicuser interface objects capable of being independently manipulated by auser of an HMD device are displayed to the user, overlaid on areal-world view of a 3D physical space in which the user is located. Inresponse to a first user action, the holographic user interface objectsare caused to appear to be combined into a holographic container objectthat appears at a first location in the 3D physical space. In responseto the first user action or a second user action, the holographiccontainer object is caused to appear to relocate to a second location inthe 3D physical space. The holographic user interface objects are thencaused to appear to deploy from the holographic container object whenthe holographic container object appears to be located at the secondlocation in the 3D physical space.

Other aspects of the technique will be apparent from the accompanyingfigures and detailed description.

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used to limit the scope of the claimed subject matter.

BRIEF DESCRIPTION OF THE DRAWINGS

One or more embodiments of the present disclosure are illustrated by wayof example and not limitation in the figures of the accompanyingdrawings, in which like references indicate similar elements.

FIG. 1 illustrates an example of an environment including an AR-HMDdevice.

FIG. 2 shows a perspective view of an example of an AR-HMD device.

FIG. 3 shows an example of a view that a user of an AR-HMD device mighthave through the device, including various real and holographic objects.

FIG. 4 shows an example of a view that the user might have through anAR-HMD device, including a holographic toolbox.

FIGS. 5A through 5E show representative images from an example of ananimation sequence of a holographic toolbox opening and deploying itscontents.

FIG. 6 shows a view similar to that of FIG. 3, but with the holographictoolbox fully opened and with its contents deployed.

FIG. 7 shows a view similar to that of FIG. 6, but from a differentviewing location of the user.

FIGS. 8A and 8B show an example of views that a user might have aftermoving the holographic toolbox to a new location.

FIG. 9 shows an example of an indicator of the holographic toolbox'slocation when the holographic toolbox is not visible to the user.

FIG. 10 shows an overall process for providing a portable holographicuser interface.

FIG. 11 shows an example of a specific implementation of the process ofFIG. 10.

FIG. 12 is a block diagram showing an example of various functionalcomponents of an AR-HMD device.

DETAILED DESCRIPTION

In this description, references to “an embodiment,” “one embodiment” orthe like, mean that the particular feature, function, structure orcharacteristic being described is included in at least one embodiment ofthe technique introduced here. Occurrences of such phrases in thisspecification do not necessarily all refer to the same embodiment. Onthe other hand, the embodiments referred to also are not necessarilymutually exclusive.

Unlike with traditional two-dimensional (2D) visual applications thatare limited to a 2D screen on a computer, tablet or smartphone device,objects in a holographic application appear to have 3D positions in thereal world. This characteristic introduces a potential problem thatholographic objects, particularly holographic graphical user interface(GUI) objects, can become difficult for the user to find or otherwiseinteract with. For example, a user of an AR-HMD device might open aholographic menu in an application while standing in his kitchen athome, and then walk into his living room. The holographic applicationmay or may not be capable of making the menu follow him around the home;if it is not, the user may or may not remember that he left the menuback in the kitchen. As another example, the user might start changingsettings for a holographic object, walk away, and then have difficultyreading the text on the holographic object settings from across theroom.

In accordance with the techniques introduced here, these and otherproblems can be solved by a holographic application that provides aportable holographic user interface. The portable holographic userinterface can include, for example, a holographic toolbox (as henceforthassumed herein to facilitate description), suitcase, or other similarcontainer object, that visually and functionally includes one or moreadditional user interface elements. In this description, a userinterface element/object or tool is a displayable object that has someuser-accessible function associated with it.

For example, upon starting the holographic application in an AR-HMDdevice, the user might “drop” the holographic toolbox in a particularroom, and it would unfold or expand into a holographic GUI of tools andobjects that a user can use to create and/or modify other holographicobjects. If the user wants to move the toolbox for any reason, he canenter a simple user input to pack up the toolbox (e.g., by using a handgesture to select the handle of the toolbox) and move the entire userinterface to a new location. In some embodiments, the holographictoolbox (or other portable holographic user interface) willautomatically follow the user around and/or will automatically maintainthe same relative position and orientation relative to the user, as theuser moves around.

These and other techniques are described further below. First, however,it is useful to describe an example of an environment and a device inwhich these techniques can be implemented.

FIG. 1 shows an example of an environment including an AR-HMD device 1that can implement the techniques introduced here. In the illustratedexample, the AR-HMD device 1 is configured to communicate data to andfrom an external processing device 2 through a connection 3, which canbe a wired connection, a wireless connection, or a combination thereof.In other use cases, however, the AR-HMD device 1 may operate as astandalone device. The connection 3 can be configured to carry any kindof data, such as image data (e.g., still images and/or full-motionvideo, including 2D and 3D images), audio data (including voice),multimedia, and/or any other type(s) of data. The processing device 2may be, for example, a game console, personal computer, tablet computer,smartphone, or other type of processing device. The connection 3 can be,for example, a universal serial bus (USB) connection, Wi-Fi connection,Bluetooth or Bluetooth Low Energy (BLE) connection, Ethernet connection,cable connection, DSL connection, cellular connection (e.g., 3G, LTE/4Gor 5G), or the like, or a combination thereof. Additionally, theprocessing device 2 may communicate with one or more other processingsystems 5 via a network 4, which may be or include, for example, a localarea network (LAN), a wide area network (WAN), an intranet, ametropolitan area network (MAN), the global Internet, or a combinationthereof.

FIG. 2 shows a perspective view of an example of an AR-HMD device thatcan implement the techniques introduced here. The AR-HMD device 20 canbe an embodiment of AR-HMD device 1 in FIG. 1. AR-HMD device 20 includesa head fitting, namely, a headband 21, by which the AR-HMD device 20 canbe worn on the user's head. Attached to the headband 21 (directly orindirectly) is a transparent protective visor 22 that encloses one ormore transparent AR display devices 23, each of which can overlayholographic images on the user's view of his real-world environment, forone or both eyes (e.g., by projecting light into the user's eyes). Theprotective visor 22 also encloses various circuitry (not shown) andsensors.

The AR-HMD device 20 further includes one or more eye-tracking cameras24, one or more microphones 25 to input speech from the user (e.g., foruse in recognizing voice commands and providing audio effects); one ormore audio speakers 26 to output sound to the user; one or morevisible-spectrum tracking cameras 27 for use in capturing images ofsurrounding surfaces to allow tracking of the user's head position andorientation in real-world space and hand gesture recognition; one ormore infrared (IR) spectrum depth cameras 28 for use in determiningdistances to nearby surfaces (e.g., for use in surface reconstruction tomodel the user's environment); one or more IR illumination sources 29for use with the depth camera(s) 28; and one or more visible spectrumvideo cameras 30 for use in capturing standard video of what the usersees. The AR-HMD device 20 also includes circuitry (not shown), whichmay be contained within the visor 22, to control at least some of theaforementioned elements and perform associated data processing functions(e.g., speech and gesture recognition and display generation). Thecircuitry may include, for example, one or more processors and one ormore memories. Note that in other embodiments the aforementionedcomponents may be located in different locations on the AR-HMD device20. Additionally, some embodiments may omit some of the aforementionedcomponents and/or may include additional components not mentioned above.

FIGS. 3 through 9 show various examples of views that a user of anAR-HMD device (or any other device that provides an immersive 3Dvisualization) might have in connection with the techniques introducedhere (e.g., through display devices 23 and visor 22 in FIG. 2). Inparticular, FIG. 3 shows an example of the view that a user of theAR-HMD device (such as AR-HMD device 20) might have while standing in aroom while wearing the device. Through the display area of the device,the user may see various real-world objects, such as the floor and wallsof the room, a chair 33, a door 35 and a table 36. The AR-HMD device mayalso display various holographic objects overlaid on the user'sreal-world view, such as a holographic ball 31 and/or a holographic cube32. These are very simple examples of holographic objects to facilitateexplanation; in practice a display system may enable the user to see andinteract with much more complex holographic objects. Note that in someinstances, the table 36 might be holographic rather than real (e.g., aholographic work surface on which to build other holographic objects).The AR-HMD device may also display one or more holographic icons 34and/or other GUI elements in the user's field of view, to enable theuser to use various functions of the AR-HMD device.

While the AR-HMD device is operational, it can use its depth camera(s)to construct a 3D mesh model of all surfaces in the user's vicinity(e.g., within several meters), or at least of all nearby surfaces withinthe user's field of view, including their distances from the user (i.e.,from the AR-HMD device). Techniques for generating a 3D mesh model ofnearby surfaces by using depth detection (e.g., time of flight) areknown in the art and need not be described herein. Accordingly, the 3Dmesh model in the example of FIG. 3 would model at least all visiblesurfaces of the chair 33, table 36 (assuming it is real and notholographic) and door 35 as well as the room's walls, floor and ceiling,windows, and potentially even smaller features such as curtains, artwork(not shown) mounted on the walls, etc. The 3D mesh model can be storedin memory on the AR-HMD device. By use of the 3D mesh model and imagedata from the visual tracking system, circuitry in the AR-HMD device canat any time determine the user's precise position within the room. The3D mesh model can be automatically updated on a frequent basis, such asseveral times per second.

By using the AR-HMD device, and through the use of hand gestures, voicecommands and/or controlled gaze, the user can create and manipulatevarious 3D holographic (AR) objects, such as sphere 31 and cube 32. Forexample, the user can create and delete holographic objects, move androtate holographic objects, change colors, fill patterns, surfacetextures and decorations of holographic objects, etc. As shown, theseobjects are overlaid on the user's view of the real world.

For example, the user may create a holographic object such as thesnowman 40 shown in FIG. 4, which appears to be resting on the surfaceof the table 36. Note that a holographic object does not actually have alocation in real (physical) space, since it is not a real object.However, in the context of this description, every holographic objecthas an apparent location in real space, which is the location in realspace at which the object appears to be located to the user. Forpurposes of this description, therefore, any reference to the “location”of a holographic object should be understood to be referring to itsapparent location to the user.

To assist the user in creating and editing holographic objects, aholographic application running on the AR-HMD device may provide aholographic user interface. In accordance with the techniques introducedhere, the holographic user interface is portable, as described furtherbelow. The portable holographic user interface may be shown initially(i.e., upon startup of the holographic application) as a singleholographic container object, such as holographic toolbox 41, asillustrated in FIGS. 4 and 5A, that “contains” one or more holographictools, which appear to deploy from it in response to a predeterminedinput or inputs as shown in FIGS. 5B through 5E. Additionally oralternatively, the portable holographic user interface could have adifferent form, such as a simple menu, button, joystick, or a much morecomplex holographic object. The example of a toolbox is only used hereinto facilitate description.

Upon startup of the application, the holographic toolbox 41 may appearto be closed, as shown in FIGS. 4 and 5A. Additionally, it may bedisplayed in any convenient location, such as floating in the air orstuck to a wall in front of the user. Preferably, the holographictoolbox 41 is displayed in a location where it does not obstruct theuser's view of (occlude) other holographic objects.

The holographic application can then cause the holographic toolbox 41 toappear to open and deploy its contents, which are tools that the usercan employ to create and edit other holographic objects. This may happenautomatically, such as when the user speaks a predetermined command(e.g., “Toolbox”), uses a hand gesture to tap an icon, drops the toolbox41 at a particular location in his 3D environment, or any otherpredetermined user input. FIGS. 5A through 5E show a sequence of imagesrepresenting how the toolbox 41 may appear to open and deploy itscontents. In some embodiments the deployment of objects from the toolbox41 may appear to the user as a movie or animation. In other embodiments,it may be a sequence of discrete steps, each requiring a specific userinput to trigger it. FIG. 6 shows an example how the toolbox 41 and itscontents might appear when fully open and deployed. As shown in FIGS. 5Eand 6, the toolbox 41 contains various different tools. These toolsmight include, for example: a magnet to pick up and move holographicobjects; a spray paint can, crayon, marker and/or other similar tool(s)for decorating holographic objects; a glue bottle for stickingholographic objects together, an eraser for deleting object features,etc. Additionally, as shown in the illustrated example, the contents ofthe toolbox 41 might include various different alphanumeric charactersand/or other characters and images, which can be applied to otherholographic objects.

In some embodiments, the holographic application will cause the toolbox41 to automatically follow the user around if the user walks away,and/or it will automatically maintain the same position and orientationof the toolbox 41 relative to the user as the user moves. For example,FIG. 7 shows an example of the view from the user's perspective afterthe user has walked partway around the table 36, relative to theperspective shown in FIG. 3: Note that the toolbox 41 in FIG. 7 appearsin the same location, relative to the direction in which the user's headis pointing, as in FIG. 3. In some embodiments, if the user moves awayfrom his current location by some predetermined amount, the holographicapplication will cause an animation sequence essentially reverse that ofFIGS. 5A through 5E to be automatically displayed, i.e., to pack up andclose the toolbox 41, and the toolbox 41 will then follow the userautomatically. FIGS. 8A and 8B show how the toolbox 41 may be displayedand redeployed in the user's new location. In some embodiments, if thetoolbox 41 is relocated by the user or is deactivated (e.g., when theuser activates another GUI feature), the various tools deployed from thetoolbox will automatically be displayed as returning to their originalpositions within the toolbox. This action may appear to the user as amovie or animation.

Performance of these functions may be made subject to certainconditions. For example, the displayed location of the toolbox 41 may beautomatically adjusted as necessary to avoid obstructing the user's viewof other holographic objects and/or to avoid having the toolbox 41 beoccluded by other objects (real or holographic). Similarly, a portion ofthe deployed toolbox 41 may be collapsed to avoid visual occlusion. Insome embodiments, the toolbox 41 may appear to remain stuck to a wall orother real object to assist the user in locating it, until and unlessthe user decides to move somewhere else.

In some embodiments, the user can pack up the toolbox 41 with a simpleuser input, such as by saying “close toolbox” or by a hand gesture to“touch” the handle of the toolbox 41. The user input will cause ananimation sequence essentially reverse that of FIGS. 5A through 5E to bedisplayed, i.e., to pack up and close the toolbox 41. The user can thenpick up the toolbox 41 (e.g., by a hand gesture), “carry” it to anotherlocation (e.g., another room), and open it up in the new location in thesame manner as described above and illustrated in FIGS. 8A and 8B.

In some embodiments, the holographic application provides visual cues tohelp the user find the toolbox 41 if the toolbox is not within theuser's current field of view. For example, the application may cause theAR-HMD device to display directional arrows or other visual aids toindicate which way the user should turn to find the toolbox 41. If thetoolbox is visually occluded by another object (real or holographic)(e.g., if it is on the other side of a wall or behind a sofa), theapplication might cause the AR-HMD to display a visual marker of thetoolbox's location behind the object, such as a “ghost” toolbox 91, asshown in FIG. 9.

FIG. 10 illustrates an overall process for providing a portableholographic user interface. The process can be performed by any displaydevice or system that provides an immersive, interactive 3Dvisualization environment, such as an AR-HMD device. For example, theprocess can be performed by an application executing in such a device.

The process of FIG. 10 may begin after various holographic userinterface objects (for example, holographic tools) have been deployedfrom a holographic container object, such as the toolbox 41 in FIGS. 6,7 and 8B. Initially, at step 1001 the process displays to the usermultiple holographic user interface objects overlaid on a real-worldview of a 3D physical space in which the user is located. One example ofsuch an arrangement of holographic objects is shown in FIG. 6, 7 or 8B,in the form of the deployed contents of the toolbox 41. The specificspatial arrangement of these objects relative to each other and to theuser can be application-specific, user-configurable, and/or dependent ona prior state of the holographic application. The holographic userinterface objects, which may be visual design tools, for example, arecapable of being independently manipulated (e.g., used, invoked oroperated) by the user. At step 1002, in response to a first prespecifieduser action, the process causes the holographic user interface objectsto appear to the user to be combined (e.g., packed or stowed) into asingle holographic container object (such as the above-describedholographic toolbox 41) that appears at a first location in the 3Dphysical space. In some embodiments step 1002 may appear to the user asa movie or animation. For example, it could appear as the various toolsand other contents of toolbox 41 automatically “flying” back into theirappropriate storage positions within the toolbox 41, e.g., according tothe reverse sequence of FIGS. 5A through 5E. The first prespecified useraction may be, for example, an action expressly intended as a userinput, such as a spoken command or hand gesture, or it may be an actionthat is not specifically intended as a user input, such as the userwalking to another room. At step 1003, in response to the firstprespecified user action or a second prespecified user action, theprocess causes the single holographic container object to appear torelocate to a second location in the 3D physical space. Thus, therelocation of the holographic container object in step 1003 may beautomatic in response to the first prespecified user action, or it maybe in response to a second prespecified user action that (like the firstprespecified user action) may or may not be expressly intended as a userinput. At step 1004, the process causes the holographic user interfaceobjects to appear to the user to deploy from the single holographiccontainer object when the single holographic container object appears tobe located at the second location in the 3D physical space.

FIG. 11 illustrates a specific implementation of the process of FIG. 10,corresponding to the holographic toolbox example of FIGS. 3 through 7.At step 1101, the process receives a prespecified user input specifyingactivation of the holographic toolbox. As noted above, this input may bea predetermined spoken command, hand gesture, or any other suitableinput. At step 1102, the process responds to that user input bydisplaying the closed holographic toolbox at a suitable location withinthe user's field of view. At step 1103 the process displays thedeployment (e.g., as an animation) of various holographic GUI tools fromthe toolbox and enables the tools for use by the user.

If and when the process detects a prespecified user action thatindicates (expressly or implicitly) an intent to move the toolbox (step1104), the process responds at step 1105 by displaying (e.g., as ananimation) the packing of the tools back into toolbox and closing of thetoolbox. The process then displays relocation of the (closed) toolboxaccording to one or more user actions, which may or may not be expresseduser inputs. The process then waits (step 1107) to receive prespecifieduser input specifying to continue use of toolbox (e.g., the spokencommand, “open toolbox,” or a hand gesture touching the handle of thetoolbox), at which point the process loops back to step 1103, describedabove.

Alternatively, instead of waiting for user input at step 1107, theprocess could immediately loop back from step 1106 to step 1103. Forexample, in one possible use scenario, the user grabs the holographictoolbox and rotates his head to indicate intent to move the toolbox ontoa real-world table. In response, the holographic toolbox immediatelycollapses, moves, then re-expands once the user's head movement hasslowed down. This approach takes advantage of the user's naturalinclination to slow down his head motion once he has successfully lookedat the target location in the real world.

Of course, many different variations of the above-described approachesare possible.

FIG. 12 shows an example of various functional components of the AR-HMDdevice 20, according to some embodiments. In FIG. 12, the functionalcomponents of the AR-HMD device 20 includes one or more instance of eachof the following: a main processor 121, memory 122, transparent displaydevice 123, depth camera 124, head tracking cameras 125, video camera126, communication device 127, and audio subsystem 128, all coupledtogether (directly or indirectly) by an interconnect 129. Theinterconnect 129 may be or include one or more conductive traces, buses,point-to-point connections, controllers, adapters, wireless links and/orother conventional connection devices and/or media, at least some ofwhich may operate independently of each other.

The main processor(s) 121 individually and/or collectively control theoverall operation of the AR-HMD device 20 and perform various dataprocessing functions. For example, the processor(s) 121 may provide orat least support the portable holographic user interface featuresdescribed above. Each processor 121 can be or include, for example, oneor more general-purpose programmable microprocessors, digital signalprocessors (DSPs), graphics processing unit (GPU), mobile applicationprocessors, microcontrollers, application specific integrated circuits(ASICs), programmable gate arrays (PGAs), or the like, or a combinationof such devices.

Data and instructions (code) 130 that configure the processor(s) 121 toexecute aspects of the technique introduced here can be stored in theone or more memories 122. Each memory 122 can be or include one or morephysical storage devices, which may be in the form of random accessmemory (RAM), read-only memory (ROM) (which may be erasable andprogrammable), flash memory, miniature hard disk drive, conventionalhard disk drive, or other suitable type of storage device, or acombination of such devices.

The depth camera(s) 124 can apply time-of-flight principles, forexample, to determine distances to nearby objects. The distanceinformation acquired by the depth camera 124 is used (e.g., byprocessor(s) 121) to construct a 3D mesh model of the surfaces in theuser's environment. The head tracking camera(s) 125 enable the AR-HMDdevice 20 to continuously track the current location and orientation ofthe user's head by acquiring images of the user's real-worldenvironment. At least some of the functionality associated with surfacedetection and head tracking may be performed by the processor(s) 121.

The communication device(s) 127 enable the AR-HMD device 20 to receivedata and/or commands from, and send data and/or commands to an externalprocessing system, such as a personal computer or game console, althoughin at least some embodiments the AR-HMD device 20 can operate as astandalone device. Each communication device 127 can be or include, forexample, a universal serial bus (USB) adapter, Wi-Fi transceiver,Bluetooth or Bluetooth Low Energy (BLE) transceiver, Ethernet adapter,cable modem, DSL modem, cellular transceiver (e.g., 3G, LTE/4G or 5G),baseband processor, or the like, or a combination thereof. The audiosubsystem 128 includes at least one speaker and audio processingcircuitry to output sound effects to the user.

The machine-implemented operations described above can be implemented byprogrammable circuitry programmed/configured by software and/orfirmware, or entirely by special-purpose circuitry, or by a combinationof such forms. Such special-purpose circuitry (if any) can be in theform of, for example, one or more application-specific integratedcircuits (ASICs), programmable logic devices (PLDs), field-programmablegate arrays (FPGAs), system-on-a-chip systems (SOCs), etc.

Software to implement the techniques introduced here may be stored on anon-transitory machine-readable storage medium and may be executed byone or more general-purpose or special-purpose programmablemicroprocessors. A “machine-readable medium,” as the term is usedherein, includes any mechanism that can store information in a formaccessible by a machine (a machine may be, for example, a computer,network device, cellular phone, personal digital assistant (PDA),manufacturing tool, any device with one or more processors, etc.). Forexample, a machine-accessible medium includes recordable/non-recordablemedia (e.g., read-only memory (ROM); random access memory (RAM);magnetic disk storage media; optical storage media; flash memorydevices; etc.), etc.

Examples of Certain Embodiments

Certain embodiments of the technology introduced herein are summarizedin the following numbered examples:

1. A method comprising: displaying to a user, by a head-mounted display(HMD) device, a plurality of holographic user interface objects overlaidon a real-world view of a 3D physical space in which the user islocated, the plurality of holographic user interface objects beingcapable of being independently manipulated the user; in response to afirst user action, causing, by the HMD device, the plurality ofholographic user interface objects to appear to the user to be combinedinto a holographic container object that appears at a first location inthe 3D physical space; in response to the first user action or a secondaction, causing, by the HMD device, the holographic container object toappear to relocate to a second location in the 3D physical space; andcausing, by the HMD device, the plurality of holographic user interfaceobjects to appear to the user to deploy from the holographic containerobject when the holographic container object appears to be located atthe second location in the 3D physical space.

2. The method of example 1, wherein the container object is moveable inthree dimensions in the 3D physical space.

3. The method of example 1 or example 2, wherein the plurality of userinterface objects represent different tools usable by the user to createor edit a holographic target object.

4. The method of any of examples 1 through 3, wherein at least one ofthe first user action or the second user action comprises a gesture or aspoken command.

5. The method of any of examples 1 through 4, wherein causing theholographic object to appear to relocate to the second location in the3D physical space is performed automatically in response to the userchanging locations within the 3D physical space.

6. The method any of examples 1 through 5, further comprising:maintaining the plurality of holographic user interface objects in aconstant orientation relative to the user while the user changeslocation or orientation within the 3D physical space.

7. The method of any of examples 1 through 6, further comprising:maintaining the plurality of holographic user interface objects at aconstant distance or within a constant range of distances from the userwhile the user changes location within the 3D physical space.

8. The method of any of examples 1 through 7, further comprising:determining locations of physical objects within the 3D physical space;wherein causing the holographic object to appear to relocate to thesecond location in the 3D physical space comprises at least one of:selecting a display location for the holographic object so as to avoidvisual occlusion of the holographic object by a physical or holographicobject; or selecting a display location for the holographic object so asto avoid visual occlusion of another holographic object by theholographic object.

9. The method of any of examples 1 through 8, further comprising:determining locations of physical objects in the 3D physical space; andcausing the container object to appear attached to a surface of aphysical object.

10. The method of any of examples 1 through 9, further comprising:determining locations of physical objects in the 3D physical space;detecting a condition that a holographic object is occluded by aphysical object, wherein the holographic object comprises at least oneof the plurality of user interface objects or the container object; andin response to detecting the condition, displaying to the user a visualcue to locate the holographic object.

11. A head-mounted display (HMD) device comprising: a head fitting toenable the head-mounted display device to be worn on the head of a user;a display device coupled to the head fitting and disposed to bepositioned in front of the eyes of the user when the HMD device is wornby the user, the display device being at least partially transparent;and a processor coupled to the display device and configured to causethe display device to display to the user a plurality of holographictools overlaid on a real-world view of a 3D physical space in which theuser is located, the plurality of holographic tools being capable ofbeing independently moved and operated by the user, each of the toolshaving a separate predetermined functionality usable by the user in aholographic workspace; in response to a first user action, cause theplurality of holographic tools to appear to the user to be combined intoa holographic container object that appears at a first location in the3D physical space; in response to the first user action or a second useraction, cause the holographic container object to appear to relocate toa second location in the 3D physical space; and cause the plurality ofholographic tools to appear to the user to deploy from the holographiccontainer object when the holographic container object appears to belocated at the second location in the 3D physical space.

12. The HMD device of example 11, wherein the plurality of holographictools are usable by the user to create or edit a holographic targetobject.

13. The HMD device of example 11 or example 12, wherein causing theholographic object to appear to relocate to the second location in the3D physical space is performed automatically in response to the userchanging locations within the 3D physical space.

14. The HMD device of any of examples 11 through 13, wherein theprocessor is further configured to cause the display device to displaythe plurality of holographic tools in a constant orientation relative tothe user while the user changes location or orientation within the 3Dphysical space.

15. The HMD device of any of examples 11 through 14, wherein theprocessor is further configured to cause the display device to displaythe plurality of holographic tools at a constant distance or range ofdistances from the user while the user changes location within the 3Dphysical space.

16. A non-transitory machine-readable storage medium storinginstructions, an execution of which by a processor causes a head-mounteddisplay (HMD) device to perform operations comprising: displaying, to auser of the HMD device, a plurality of holographic user interfaceobjects overlaid on a real-world view of a 3D physical space in whichthe user is located, each of the plurality of user interface objectsrepresenting a different tool independently usable by the user to createor edit a holographic target object; in response to a first user inputfrom the user, causing the plurality of holographic user interfaceobjects to appear to the user to be combined into a holographiccontainer object that appears at a first location in the 3D physicalspace, wherein the container object is moveable in three dimensions inthe 3D physical space; in response to a second user input from the userindicating a relocation action, causing the holographic container objectto appear to relocate to a second location in the 3D physical space; andcausing the plurality of holographic user interface objects to appear tothe user to deploy from the holographic container object when theholographic container object appears to be located at the secondlocation in the 3D physical space.

17. The non-transitory machine-readable storage medium of example 16,wherein the plurality of holographic tools are usable by the user tocreate or edit a holographic target object.

18. The non-transitory machine-readable storage medium of example 16 orexample 17, wherein causing the holographic object to appear to relocateto the second location in the 3D physical space is performedautomatically in response to the user changing locations within the 3Dphysical space.

19. The non-transitory machine-readable storage medium of any ofexamples 16 through 18, the operations further comprising: causing thedisplay device to display the plurality of holographic tools in aconstant orientation relative to the user while the user changeslocation or orientation within the 3D physical space.

20. The non-transitory machine-readable storage medium of any ofexamples 16 through 19, the operations further comprising: causing thedisplay device to display the plurality of holographic tools at aconstant distance or range of distances from the user while the userchanges location within the 3D physical space.

21. An apparatus comprising: means for displaying, to a user of adisplay device, a plurality of holographic user interface objectsoverlaid on a real-world view of a 3D physical space in which the useris located, the plurality of holographic user interface objects beingcapable of being independently manipulated the user; means for causing,in response to a first user action, the plurality of holographic userinterface objects to appear to the user to be combined into aholographic container object that appears at a first location in the 3Dphysical space; means for causing, in response to the first user actionor a second action, the holographic container object to appear torelocate to a second location in the 3D physical space; and means forcausing the plurality of holographic user interface objects to appear tothe user to deploy from the holographic container object when theholographic container object appears to be located at the secondlocation in the 3D physical space.

22. The apparatus of example 21, wherein the container object ismoveable in three dimensions in the 3D physical space.

23. The apparatus of example 21 or example 22, wherein the plurality ofuser interface objects represent different tools usable by the user tocreate or edit a holographic target object.

24. The apparatus of any of examples 21 through 23, wherein at least oneof the first user action or the second user action comprises a gestureor a spoken command.

25. The apparatus of any of examples 21 through 24, wherein the meansfor causing the holographic object to appear to relocate to the secondlocation in the 3D physical space causes the holographic object toappear to relocate automatically in response to the user changinglocations within the 3D physical space.

26. The apparatus any of examples 21 through 25, further comprising:means for maintaining the plurality of holographic user interfaceobjects in a constant orientation relative to the user while the userchanges location or orientation within the 3D physical space.

27. The apparatus of any of examples 21 through 26, further comprising:means for maintaining the plurality of holographic user interfaceobjects at a constant distance or within a constant range of distancesfrom the user while the user changes location within the 3D physicalspace.

28. The apparatus of any of examples 21 through 27, further comprising:means for determining locations of physical objects within the 3Dphysical space; wherein causing the holographic object to appear torelocate to the second location in the 3D physical space comprises atleast one of: selecting a display location for the holographic object soas to avoid visual occlusion of the holographic object by a physical orholographic object; or selecting a display location for the holographicobject so as to avoid visual occlusion of another holographic object bythe holographic object.

29. The apparatus of any of examples 21 through 28, further comprising:means for determining locations of physical objects in the 3D physicalspace; and causing the container object to appear attached to a surfaceof a physical object.

30. The apparatus of any of examples 21 through 29, further comprising:means for determining locations of physical objects in the 3D physicalspace; detecting a condition that a holographic object is occluded by aphysical object, wherein the holographic object comprises at least oneof the plurality of user interface objects or the container object; andin response to detecting the condition, displaying to the user a visualcue to locate the holographic object.

Any or all of the features and functions described above can be combinedwith each other, except to the extent it may be otherwise stated aboveor to the extent that any such embodiments may be incompatible by virtueof their function or structure, as will be apparent to persons ofordinary skill in the art. Unless contrary to physical possibility, itis envisioned that (i) the methods/steps described herein may beperformed in any sequence and/or in any combination, and that (ii) thecomponents of respective embodiments may be combined in any manner.

Although the subject matter has been described in language specific tostructural features and/or acts, it is to be understood that the subjectmatter defined in the appended claims is not necessarily limited to thespecific features or acts described above. Rather, the specific featuresand acts described above are disclosed as examples of implementing theclaims and other equivalent features and acts are intended to be withinthe scope of the claims.

What is claimed is:
 1. A method comprising: displaying to a user, by ahead-mounted display (HMD) device, a plurality of holographic userinterface objects overlaid on a real-world view of a three-dimensional(3D) physical space in which the user is located, wherein each of theholographic user interface objects is a different tool usable by theuser to affect a holographic target object, each of the plurality ofholographic user interface objects being capable of being independentlymoved and used by the user; in response to a first user action, causing,by the HMD device, the plurality of holographic user interface objectsto appear to the user to be physically placed into a 3D holographiccontainer object that appears at a first location in the 3D physicalspace; in response to the first user action or a second user action,causing, by the HMD device, the 3D holographic container object toappear to relocate to a second location in the 3D physical space; andcausing, by the HMD device, the plurality of holographic user interfaceobjects to appear to the user to deploy from within the 3D holographiccontainer object when the 3D holographic container object appears to belocated at the second location in the 3D physical space.
 2. The methodof claim 1, wherein the 3D holographic container object is moveable inthree dimensions in the 3D physical space.
 3. The method of claim 1,wherein at least one of the first user action or the second user actioncomprises a gesture or a spoken command.
 4. The method of claim 1,wherein causing the 3D holographic object to appear to relocate to thesecond location in the 3D physical space is performed automatically inresponse to the user changing locations within the 3D physical space. 5.The method of claim 1, further comprising: maintaining the plurality ofholographic user interface objects in a constant orientation relative tothe user while the user changes location or orientation within the 3Dphysical space.
 6. The method of claim 1, further comprising:maintaining the plurality of holographic user interface objects at aconstant distance or within a constant range of distances from the userwhile the user changes location within the 3D physical space.
 7. Themethod of claim 1, further comprising: determining locations of physicalobjects within the 3D physical space; wherein causing the holographicobject to appear to relocate to the second location in the 3D physicalspace comprises at least one of: selecting a display location for theholographic object so as to avoid visual occlusion of the holographicobject by a physical or holographic object; or selecting a displaylocation for the holographic object so as to avoid visual occlusion ofanother holographic object by the holographic object.
 8. The method ofclaim 1, further comprising: determining locations of physical objectsin the 3D physical space; and causing the 3D holographic containerobject to appear attached to a surface of a physical object.
 9. Themethod of claim 1, further comprising: determining locations of physicalobjects in the 3D physical space; detecting a condition that aholographic object is occluded by a physical object, wherein theholographic object comprises at least one of the plurality of userinterface objects or the 3D holographic container object; and inresponse to detecting the condition, displaying to the user a visual cueto locate the holographic object.
 10. A head-mounted display (HMD)device comprising: a head fitting to enable the head-mounted displaydevice to be worn on the head of a user; a display device coupled to thehead fitting and disposed to be positioned in front of the eyes of theuser when the HMD device is worn by the user, the display device beingat least partially transparent; and a processor coupled to the displaydevice and configured to cause the display device to display to the usera plurality of holographic tools overlaid on a real-world view of athree-dimensional (3D) physical space in which the user is located,wherein each of the holographic tools is usable by the user to affect aholographic target object, each of the plurality of holographic toolsbeing capable of being independently moved and operated by the user,each of the tools having a separate predetermined functionality usableby the user in a holographic workspace; in response to a first useraction, cause the plurality of holographic tools to appear to the userto be physically placed into a 3D holographic container object thatappears at a first location in the 3D physical space; in response to thefirst user action or a second user action, cause the 3D holographiccontainer object to appear to relocate to a second location in the 3Dphysical space; and cause the plurality of holographic tools to appearto the user to deploy from within the 3D holographic container objectwhen the 3D holographic container object appears to be located at thesecond location in the 3D physical space.
 11. The HMD device of claim10, wherein causing the holographic object to appear to relocate to thesecond location in the 3D physical space is performed automatically inresponse to the user changing locations within the 3D physical space.12. The HMD device of claim 10, wherein the processor is furtherconfigured to cause the display device to display the plurality ofholographic tools in a constant orientation relative to the user whilethe user changes location or orientation within the 3D physical space.13. The HMD device of claim 10, wherein the processor is furtherconfigured to cause the display device to display the plurality ofholographic tools at a constant distance or range of distances from theuser while the user changes location within the 3D physical space.
 14. Anon-transitory machine-readable storage medium storing instructions, anexecution of which by a processor causes a head-mounted display (HMD)device to perform operations comprising: displaying, to a user of theHMD device, a plurality of holographic user interface objects overlaidon a real-world view of a three-dimensional (3D) physical space in whichthe user is located, each of the plurality of holographic user interfaceobjects being a different tool independently movable and usable by theuser to create or edit a holographic target object; in response to afirst user input from the user, causing the plurality of holographicuser interface objects to appear to the user to be physically placedinto a 3D holographic container object that appears at a first locationin the 3D physical space, wherein the 3D holographic container object ismoveable in three dimensions in the 3D physical space; in response to asecond user input from the user indicating a relocation action, causingthe 3D holographic container object to appear to relocate to a secondlocation in the 3D physical space; and causing the plurality ofholographic user interface objects to appear to the user to deploy fromwithin the 3D holographic container object when the 3D holographiccontainer object appears to be located at the second location in the 3Dphysical space.
 15. The non-transitory machine-readable storage mediumof claim 14, wherein causing the holographic object to appear torelocate to the second location in the 3D physical space is performedautomatically in response to the user changing locations within the 3Dphysical space.
 16. The non-transitory machine-readable storage mediumof claim 14, the operations further comprising: causing the displaydevice to display the plurality of holographic tools in a constantorientation relative to the user while the user changes location ororientation within the 3D physical space.
 17. The non-transitorymachine-readable storage medium of claim 14, the operations furthercomprising: causing the display device to display the plurality ofholographic tools at a constant distance or range of distances from theuser while the user changes location within the 3D physical space.